Skip to content

fix(ebpf): synchronize perfUsageCollector.Collect with Manager.Start - #54952

Draft
pgimalac wants to merge 1 commit into
mainfrom
pgimalac/fix-perf-usage-collector-race
Draft

fix(ebpf): synchronize perfUsageCollector.Collect with Manager.Start#54952
pgimalac wants to merge 1 commit into
mainfrom
pgimalac/fix-perf-usage-collector-race

Conversation

@pgimalac

Copy link
Copy Markdown
Member

What does this PR do?

Fixes a data race on PerfMap/RingBuffer telemetry fields: perfUsageCollector.Collect reads BufferSize() (which reads bufferSize without a lock) during Prometheus Gather ticks, while ebpf.Manager.Start writes those same fields during eBPF program startup. Changes the collector mutex from sync.Mutex to sync.RWMutex so Collect takes RLock (reader), and exports LockForWrite/UnlockForWrite so Manager.Start can take the write lock around m.Manager.Start().

Motivation

Fix a race, found via a race-detector-enabled build in staging (see #54333 for context).

Describe how you validated your changes

Added TestLockForWriteBlocksCollect and TestCollectConcurrentReaders which validate the lock mechanism: LockForWrite blocks Collect until UnlockForWrite is called, and multiple Collect calls can run concurrently under RLock. Tests pass under -race (dda inv test --targets=./pkg/ebpf/telemetry/... --race). The actual race requires eBPF support to trigger (Manager.Start writes to PerfMap/RingBuffer internal fields), so the test validates the synchronization mechanism rather than the full race.

@pgimalac pgimalac added qa/done QA done before merge and regressions are covered by tests changelog/no-changelog No changelog entry needed labels Aug 17, 2026
@github-actions

Copy link
Copy Markdown
Contributor

@codex review

@dd-octo-sts dd-octo-sts Bot added internal Identify a non-fork PR team/ebpf-platform labels Aug 17, 2026
@github-actions github-actions Bot added the short review PR is simple enough to be reviewed quickly label Aug 17, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6ac15534c1

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread pkg/ebpf/manager.go
Comment thread pkg/ebpf/telemetry/perf_metrics.go Outdated
Comment thread pkg/ebpf/telemetry/perf_metrics_test.go Outdated
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 52.45% (+0.05%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 4b92898 | Docs | Datadog PR Page | Give us feedback!

perfUsageCollector.Collect reads PerfMap/RingBuffer telemetry fields
(e.g. BufferSize) during Prometheus Gather ticks, while ebpf.Manager.Start
writes those same fields during eBPF program startup. The collector's
mutex protected its own data structures but not the shared PerfMap/
RingBuffer objects.

Change the collector mutex from sync.Mutex to sync.RWMutex so Collect
takes RLock (reader), and export LockForWrite/UnlockForWrite so
Manager.Start can take the write lock around m.Manager.Start(). This
prevents Collect from reading telemetry fields while Start is writing them.

Found via a race-detector-enabled build in staging (see #54333).
@pgimalac
pgimalac force-pushed the pgimalac/fix-perf-usage-collector-race branch from 6ac1553 to 4b92898 Compare August 17, 2026 11:17
@github-actions github-actions Bot added medium review PR review might take time and removed short review PR is simple enough to be reviewed quickly labels Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/no-changelog No changelog entry needed component/system-probe internal Identify a non-fork PR medium review PR review might take time qa/done QA done before merge and regressions are covered by tests team/agent-build team/ebpf-platform

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant